(MOT-4358) feat(provider-groq): add the Groq provider worker - #712

…e provider scaffold

Review cleanups, wire-identical (goldens unchanged in all three
crates). The byte-identical openai and codex counters collapse into
llm_router::provider_scaffold::tiktoken_count with the framing
constants, encoder selection, and the shared test suite (including the
cases one copy had dropped); each provider keeps a thin request
adapter and the tiktoken dependency moves to the one crate. embed now
detects a missing provider function with the same typed helper
count_tokens uses instead of string matching.

…ounting

…wn vocabulary

Counting a DeepSeek model with tiktoken would be wrong in a way nobody
could see: an OpenAI-compatible wire shape does not imply an OpenAI
vocabulary, so the number would look authoritative while being off by
whatever the two disagree about. DeepSeek publishes no metering endpoint
but does publish its tokenizer, so the count is computed from that.

`provider_scaffold::vocabulary_count` fetches a vocabulary once, caches it
under `~/.iii/tokenizers/` behind an atomic rename so a killed process
cannot poison the cache, and parses it once per process. Resolving it at
runtime rather than compiling a table in is what lets a model announced
tomorrow count correctly today. A cold cache with no network returns the
typed `no_token_counter` error, leaving the caller on its own estimate
rather than reporting a wrong number as exact.

`chat_framing` carries the parts every local counter shares — which text a
message contributes, what the framing costs, how a tool schema serializes
— so tiktoken and vocabulary counting cannot drift apart, and a third
tokenizer is a closure rather than a third copy of these rules.

tokenizers is pinned with default features off: they pull onig (C) and
esaxx (C++), which would need a cross C toolchain on all nine release
targets.

Measured against DeepSeek's own billed usage on a live rig: 8938 counted,
8938 billed. The heuristic it replaces was reporting a 8416-token system
prompt as 5091.

Four providers, three different truths about who owns the tokenizer, so
the seam is drawn where the difference actually is.

Moonshot and llama.cpp meter a prompt themselves, so the count is simply
asked for: Moonshot through its estimator endpoint, llama.cpp through the
Anthropic-compatible count route it already speaks. llama.cpp is the one
that could not have been solved any other way — the operator loads
whichever GGUF they like, and no table compiled into this binary could
know which vocabulary sits in memory right now.

xAI splits the difference: it publishes a tokenizer but not a prompt
meter, so xAI owns the vocabulary and this worker owns the chat framing.
The request is tokenized in one call rather than one per row, which costs
a separator token per join, and xAI's own FAQ notes the tokenizer can
disagree with billing.

Z.AI publishes neither, but GLM's vocabulary is public, so it counts the
way DeepSeek does. Borrowing tiktoken here would have been worst of all:
GLM's vocabulary disagrees with it most on the Chinese text these models
are used for.

`endpoint_count` carries what the metered providers share — a bounded
timeout, a status check that keeps the upstream's own words, and pulling a
number out of a reply whose shape nobody agrees on. `chat_framing` gains
`frame`, splitting "what gets counted" from "how it adds up" so a remote
tokenizer can batch a whole request into one call.

Estimator strings now say which kind of answer a count is: `metered` when
the upstream produced it, `tokenizer` when a real vocabulary did locally.

…not one segment up

Verifying kimi against Moonshot caught the bug: `chat/completions` is two
path segments, so cutting one built
`…/v1/chat/tokenizers/estimate-token-count`, which no upstream serves. xAI
had it too — both default to a `/v1/chat/completions` endpoint. Routes now
hang off the API base the same way each provider's discovery already
derives its models route, with tests naming the wrong URL so it stays out.

Both counting endpoints are now verified against real servers rather than
documentation. Moonshot answers `{"data":{"total_tokens":N}}`, and returns
9 tokens for kimi-k2.5 against 87 for kimi-k3 on a byte-identical body —
model-side overhead no local estimate could have known about, which is the
argument for metered counting in one number. llama.cpp answers
`{"input_tokens":N}` on its Anthropic-compatible route, verified against a
running llama-server; its URL derivation is now tested too, since that is
the half a golden cannot check.

Groq is an inference host rather than a model vendor, and that is the whole
difference. Every provider before it serves one family, so one set of
answers held for the provider as a whole. Here a Llama, a GPT-OSS and a Qwen
model sit behind one endpoint, and three things stop being provider-wide
facts:

Counting picks the vocabulary per model. A single fixed one would be wrong
for most of the catalog, and borrowing tiktoken for all of it would be wrong
quietly — the number would read as authoritative while being off by whatever
the vocabularies disagree about. A model no rule recognizes gets the typed
`no_token_counter` instead of a guess, which leaves the caller exactly where
it would have been without the function. Meta's repositories are gated
behind a licence click a worker cannot perform, so Llama resolves through a
public mirror of the identical tokenizer.

Reasoning is per model: the GPT-OSS models take `reasoning_effort`, the
Llama models do not reason at all, so the catalog marks it per row. Groq has
no `thinking` object to enable first, and its ladder stops at `high`, so
`xhigh` saturates rather than inventing a tier the API would reject.

The catalog is mostly Groq's own: `GET /models` reports `context_window` and
`active` per model, both taken live, so a window Groq raises arrives without
a release and a model that cannot serve is never offered. Speech models
share that listing and are dropped — the absence of a context window is what
tells them apart — while a gateway that reports no windows at all is left
alone, since requiring the field there would empty the catalog.

Pricing comes from published third-party tracking: Groq's own pricing page
renders client-side and ships no figures in the document. Noted in the code
so nobody mistakes it for a vendor source.

Stacked on MOT-4329, whose vocabulary scaffold this uses. No live
verification yet — there is no Groq key on the rig, and everything else in
this series was verified at the wire before shipping.

Verifying against the live API rewrote most of what the docs implied.

`GET /models` turns out to carry the display name, the context window, the
output ceiling, the modalities, the supported features and live per-token
pricing. So the hand-kept table is gone: keeping a local price list beside a
live one is strictly worse, because it goes stale in silence. What is left
locally is the floor a sparse row falls back to, and it now claims no
capability it has not been told about — a host serving other people's
models has no provider-wide answer to "does this take tools", and
`llama-3.1-8b-instant` and `allam-2-7b` genuinely disagree.

Two things the docs got wrong and the wire did not:

`whisper-large-v3` reports a context window like everything else (448), so
the rule that dropped speech models by the absence of one dropped nothing
and would have put Whisper in the model picker. Modality is what separates
them, and that is what the filter reads now.

`qwen/qwen3.6-27b` accepts images. The catalog declared no Groq model could,
because at a single-family provider that was a provider-wide fact worth
hardcoding. Here it is per model, read from `input_modalities`.

Also live: a prompt over the per-minute token budget comes back as HTTP 413
carrying `code: rate_limit_exceeded`. The status alone reads as "too large
for the model", which would send the router off to compact a prompt that was
never too large — it was too large this minute. The envelope code wins over
the status now, with the captured body as the test.

Prices are scaled from per-token strings, and rounded: 0.00000079 times a
million is 0.7899999999999999 in binary floating point, which was reaching
the catalog verbatim.

Verified on the rig: 11 chat models discovered from 15 rows, speech dropped,
capabilities and pricing live, and per-model vocabularies counting through
three different tokenizers with a typed refusal for the family that has
none.

…into groq-update